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SUMMARY  PAGE 


PROBLEM 

A  question  of  frequent  interest  to  the  military  is  whether  some  aspect 
of  the  environment  (e.g.,  high  altitude,  extreme  temperature,  undersea 
compression,  vibration,  ship  motion)  affects  human  performance  capability. 
However,  only  those  capabilities  required  by  military  jobs  are  relevant  to 
the  question.  Hence,  a  problem  which  must  be  solved  before  the  question 
can  be  answered  is  to  find  out  what  capabilities  are  required  by  military 
jobs  to  be  performed  in  unusual  military  environments. 


FINDINGS 

A  job-analysis  questionnaire  was  responded  to  by  16  incumbent  Aegis 
combat  system  submode  operators.  Their  responses  were  the  basis  for  analyses 
related  to  commonality  among  submodes,  job  elements,  and  sequences  typical 
of  submode  operation,  and  the  educational  and  training  requirements  of  the 
submodes.  It  is  concluded  that  there  is  considerable  commonality  among  the 
submodes.  Job  elements  typical  of  submode  operation  involve  capabilities 
for  communication,  alert  watchstanding,  and  attending  to  detail  while  work¬ 
ing  under  distractions.  Capabilities  for  writing,  use  of  mathematics,  and 
reasoning  were  reported  to  be  relatively  unimportant  for  submode  operation. 
Typical  sequences  of  job  elements  end  with  use  of  activation  controls  (e.g., 
VAB) ,  or  verbal  orders/instruction/requests,  preceded  by  the  use  of  visual 
displays  or  decision  making  without  reference  to  an  external  information 
source.  Educational  requirements  for  submode  operation  were  a  high-school 
education,  on  the  average.  Perceived  training  time  required  was  extremely 
variable  with  a  mean  of  about  eight  weeks.  About  10  hours  of  study  per 
month  was  perceived  to  be  necessary  to  maintain  proficiency  in  operating 
procedures,  doctrine,  etc. 


RECOMMENDATIONS 

Use  tests  of  communication,  alert  watchstanding,  and  attending  to 
detail  while  working  under  distractions  and  de-emphasize  tests  of  writing, 
mathematics,  and  reasoning  in  experiments  intended  to  he  applied  to  AEGIS- 
like  military  jobs. 


The  work  was  funded  by  the  Naval  Medical  Research  and  Development  Command. 

The  volunteers  used  in  this  study  were  recruited,  evaluated,  and  employed  in 
accordance  with  the  procedures  specified  in  the  Secretary  of  the  Navy 
Instruction  3900.39  series  and  the  Bureau  of  Medicine  and  Surgery  Instruction 
3900.6  series.  These  instructions  are  based  upon  voluntary  consent,  and  meet 
or  exceed  the  provisions  of  prevailing  national  and  International  guidelines. 
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INTRODUCTION 

This  is  the  report  of  a  job  analysis  of  the  Aegis  Combat  System  submode 
operator's  job.  Job  analysis  is  the  process  of  obtaining  and  summarizing  job- 
related  information  that  can  be  used  as  a  basis  for  equipment  design,  personnel 
placement,  training,  or  remuneration.  The  Aegis  Combat  System  is  a  recent 
improvement  of  the  traditional  Navy  combat  Information  center,  which  has  the 
function  of  detecting,  tracking,  identifying,  and  (if  necessary)  engaging  man¬ 
made  objects  within  range  of  its  sensors.  The  Aegis  Combat  System  includes 
several  operator  work  stations  called  subraodes.  Ostensibly,  the  submodes  are 
quite  similar,  although  each  has  a  different  specialized  function.  One 
objective  of  this  job  analysis  was  to  assess  the  degree  of  similarity  of  the 
jobs  of  the  submode  operators.  Given  a  high  degree  of  similarity,  a  second 
objective  was  to  describe  the  job  elements  characteristic  of  Aegis  submodes  in 
general.  The  third  objective  was  to  develop  a  job  analysis  method  for 
obtaining  information  about  natural  sequences  of  job  activities,  and  to  use  the 
method  to  describe  such  sequences  in  the  operation  of  Aegis  submodes.  Finally, 
the  analysis  was  intended  to  provide  information  about  the  educational  and 
training  requirements  of  Aegis  submodes. 

The  present  job  analysis  of  Aegis  submodes  was  preceded  by  another  analysis 
(Shannon  &  Carter,  1981).  That  analysis  was  based  upon  the  opinions  of  two  job 
analysts  who  used  the  Position  Analysis  Questionnaire  (PAQ  Services,  Logan,  Utah, 
84321)  for  analysis  of  the  submode  operator’s  job.  According  to  its  author 
(McCormick,  1979)  ,  the  PAQ  "is  a  structured  job  analysis  questionnaire  that 
provides  for  analyzing  jobs  in  terms  of  187  job  elements.  The  elements  are  of 
a  worker-oriented  nature  that  tend  to  characterize,  or  to  imply,  the  human 
behaviors  that  are  involved  in  jobs.”  Information  from  a  sample  of  PAQ’s  on 
2,200  jobs  from  thoughout  the  world  of  work  was  used  to  generate  some  basic 
dimensions  representing  all  occupations  (McCormick,  1979).  The  earlier 
analysis  by  Shannon  and  Carter  (1981)  showed  which  of  those  dimensions  were 
most  characteristic  of  the  Aegis  submode  jobs.  The  most  important  job  ele¬ 
ments  of  those  characteristic  dimensions  were  also  identified.  In  the  present 
analysis,  incumbent  subraode  operators  responded  to  PAQ  questions  about  job 
elements  shown  to  be  especially  relevant  to  Aegis  by  the  preceding  job  analysis. 

All  submodes,  except  those  related  to  anti-submarine  warfare,  were  represented. '  v 
The  PAQ  questions  to  which  the  incumbents  responded  were  rewritten  with  Navy  ^ 

examples  and  assembled  into  a  questionnaire  (Appendix  A). 

METHOD 


Aegis  Questionnaire 

The  first  section  of  the  questionnaire  consisted  of  four  questions  about 
training  and  educational  requirements  of  Aegis  submodes.  It  also  included  a 
question  on  the  level  of  responsibility  of  the  operator  for  the  safety  of 
others. 

The  second  section  of  the  questionnaire  consisted  of  31  PAQ  questions 
found  in  a  previous  analysis  (Shannon  &  Carter,  1981)  to  be  especially  relevant 
to  Aegis  subraodes.  Three  5-point  scales  were  used  by  the  incumbents  to  respond 
to  each  question.  The  first  scale  was  the  importance  to  the  operation  of  the 
incumbent’s  submode  of  the  lob  element  described  by  the  question.  This  is  the 
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scale  used  on  the  PAQ.  One  additional  scale  was  the  frequency  of  the  job 
element.  Frequency  could  be  high  or  low,  independent  of  importance.  A  second 
additional  scale  was  the  difficulty  of  the  job  element.  Difficulty  of  doing 
the  element  once  could  be  high  or  low,  independent  of  frequency  or  importance. 
The  two  additional  scales  were  included  to  discover  whether  they  could  add  any¬ 
thing  to  the  information  provided  by  the  importance  scale.  It  was  reasoned  that 
frequency  and  difficulty  have  Implications  for  reliability,  personnel  selection, 
training,  and  equipment  design  that  cannot  be  gleaned  from  the  importance  of 
the  job  elements. 

The  final  section  of  the  questionnaire  solicited  information  about  the 
sequence  of  job  elements.  For  example,  perhaps  decision  making  is  often 
followed  by  verbal  communication.  In  order  to  show  such  sequences,  numbers 
for  each  of  the  31  job  elements  in  the  questionnaire  were  listed  and  followed 
by  a  blank.  In  the  blank,  the  respondent  was  to  put  the  number  of  the  job 
element  (if  any)  which  usually  followed  the  element  being  considered.  This 
information  was  used  to  generate  a  job-element  transition  matrix  to  show  the 
number,  Nij ,  of  respondents  indicating  that  element  i  was  followed  by  job 
element  j . 

Respondents 


The  respondents  were  16  incumbent  submode  operators  with  recent  intense 
experience  operating  their  submodes.  Although  this  job  analysis  was  planned 
to  be  as  representative  as  possible  of  the  Aegis  submode  operators’  job, 
several  characteristics  of  the  respondents  may  limit  the  generalizability  of 
the  present  results  to  other  Aegis  contexts.  First,  the  incumbents  were  par¬ 
ticipants  in  a  one-week  training  exercise.  The  job  of  learning  to  operate 
the  Aegis  submodes  may  be  somewhat  different  from  operation  in  a  combat-ready 
status.  Furthermore,  the  incumbents  were  a  mixture  of  trainees  and  more 
experienced  trainers,  which  may  not  be  a  typical  situation.  Also,  the  Aegis 
system  operated  by  our  respondents  was  unusual  in  that  it  was  installed  in  a 
cruiser  superstructure  built  on  land  for  test  and  evaluation  purposes. 
Finally,  the  present  results  may  not  be  representative  because  of  the  arti¬ 
ficial  nature  of  the  threat  situation  posed  by  frequent  air  sorties  by 
friendly  forces  against  the  Aegis  system  trainer.  These  limitations  of  our 
data  should  be  kept  in  mind  when  evaluating  the  results.  Eight  of  the 
incumbents  operated  coordinator  submodes  (e.g..  Anti  Air  Warfare  Coordinator, 
Combat  Systems  Coordinator,  Tactical  Information  Coordinator,  Surface  Warfare 
Coordinator,  and  Test  Control  Monitor),  and  the  other  eight  were  subordinate 
(Radar  Systems  Controller,  Air  Intercept  Controller,  Identification  Super¬ 
visor,  Air  Control  Supervisor,  Missile  Systems  Supervisor,  and  Air  Radar 
Controller) .  It  seemed  reasonable  that  if  there  were  a  difference  in  the 
job  elements  of  the  various  submodes  that  it  would  show  up  most  clearly 
in  a  contrast  of  coordinator  and  subordinate  submodes. 

Data  from  the  earlier  job  analysis  (Shannon  &  Carter,  1981)  of  Aegis 
submodes  was  available  to  be  contrasted  with  data  from  the  incumbents.  This 
data  will  be  used  to  answer  the  question  of  whether  the  job  analysts  and  the 
Incumbents  had  a  similar  interpretation  of  the  job. 
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RESULTS 


The  results  will  be  presented  within  the  framework  of  several  questions 
which  motivated  the  analysis.  Did  incumbents  give  similar  descriptions  of 
their  jobs?  Did  the  job  analysts  and  the  incumbents  agree  in  their  des¬ 
cription  of  the  job?  Can  Frequency  and  Difficulty  scales  on  a  job  analysis 
questionnaire  add  to  information  obtained  from  the  usual  Importance  scale? 
What  were  the  job  elements  which  characterized  the  Aegis  submodes?  What  are 
the  typical  sequences  of  job  elements  in  operation  of  submodes?  How  much 
time  have  operators  spent  in  formal  education,  training,  and  updating  job 
knowledge? 

Agreement  Between  Analysts  and  Incumbents,  and  Among  Incumbents 

Agreement  between  analysts  and  incumbents  would  be  indicated  if  the 
sane  job  elements  were  rated  high  or  low  by  both  analysts  and  incumbents. 

Such  a  condition  is  indicated  by  large  correlations  (near  1.0)  among  analysts’ 
and  incumbents’  ratings.  Table  1  shows  the  correlations  among  incumbents* 
and  analysts’  ratings.  Ratings  are  compared  from  each  of  the  two  types  of 
incumbents  (coordinators  and  subordinates),  for  each  of  the  three  types  of 
rating  scales  (frequency,  importance,  and  difficulty).  All  these  were  also 
compared  with  the  analysts*  ratings  of  Aegis  submodes,  the  naval  career 
specialty  of  Operations  Specialist,  and  a  similar  job  performed  by  the 
Combat  Information  Control  Officer  aboard  the  E-2  advanced  early  warning 
aircraft. 

There  was  a  pattern  of  similarity  between  the  incumbents’  Difficulty 
ratings  (columns  3  and  6)  and  the  analysts’  Importance  ratings  (rows  7,  8, 
and  9).  The  absence  of  large  (near  1.0  correlations)  in  the  last  three  rows 
of  columns  2  and  5  of  Table  1  Indicates  that  the  job  analysts  and  the  Incum¬ 
bents  did  not  agree  much  on  the  relative  importance  of  the  job  elements  on 
this  questionnaire.  Some  possible  reasons  for  this  lack  of  agreement  are 
differences  of  interpretation  of  the  importance  scale  by  incumbents  and 
analysts,  differences  in  intended  ratings  of  importance,  or  the  fact  that 
the  anaysts  were  rating  these  31  job  elements  in  the  context  of  the  187 
elements  of  the  PAQ  whereas  the  incumbents  were  rating  these  31  elements  in 
isolation. 

Principal  components  (PC)  analysis  of  correlation  tables  like  Table  1 
often  serve  to  highlight  relationships  like  those  discussed  in  the  preceeding 
paragraph.  Table  2  summarizes  a  PC  analysis.  The  two  factors  listed  in  the 
table  explain  75%  of  the  standardized  variability  of  the  incumbents’  and 
analysts’  ratings.  The  first  factor  explains  43%  of  the  variance,  and 
factors  beyond  the  second  add  no  information.  Factor  1  represents  a  strong 
relationship  between  coordinators’  and  subordinates’  Frequency  and  Importance 
ratings.  Factor  2  represents  a  strong  relationship  among  the  analysts’ 
Importance  ratings  of  three  jobs  (Aegis,  OS,  and  E-2).  The  incumbents’ 
Difficulty  ratings  appear  to  be  somewhat  related  to  the  analysts'  Importance 
ratings. 

Did  the  analysts  agree  with  incumbents  in  their  description  of  the  job? 
The  analysts  appear  to  have  given  the  same  interpretation  to  Importance  that 
the  incumbents  gave  to  Difficulty.  This  finding  is  difficult  to  explain. 


Table  1 

Correlation  Matrix  of  Mean  Task  Analysis  Questionnaire  Ratings  of 
Operators  (on  three  scale  types)  and  Job  Analysts  (for  three  positions) 
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In  contrast,  the  coordinator  and  subordinate  incumbents  appear  to  have 
almost  complete  agreement  about  the  Frequency /Importance  of  the  elements  of 
their  job.  In  no  aspect  of  the  results  summarized  in  Tables  1  and  2  is  it 
possible  to  differentiate  the  responses  of  incumbents  in  coordinator  subraodes 
from  those  of  incumbents  in  subordinate  subraodes. 


Table  2 

Rotated  Factor  Loadings  of  Operators*  Ratings  (on  3  scales) 
and  Job  Analysts'  Ratings  (on  3  different  positions) 


Factor  1 

Factor  2 

Frequency 

(Com /Coord) 

1 

-.929 

-.065 

Importance 

(Com /Coord) 

2 

-.902 

.113 

Difficulty 

(Com/Coord) 

3 

.311 

.693 

Frequency 

(Subordinate) 

4 

-.894 

-.101 

Importance 

(Subordinate) 

5 

-.839 

.205 

Dif f iculty 

(Subordinate) 

6 

.453 

.674 

Job  Analyst 

(Aegis) 

7 

.026 

.881 

Job  Analyst 

(E2) 

8 

-.061 

.925 

Job  Analyst 

(OS) 

9 

-.254 

.756 

If  instead  of  considering  correlations  we  consider  the  average  ratings, 
coordinators  rated  their  jobs  higher  in  Frequency,  Importance,  and  Difficulty 
than  subordinates  (the  statistical  test  was  £(1,26)  *  4.56,  j)  =  .04). 

However,  the  pattern  of  average  Frequency,  Importance,  and  Difficulty  ratings 
was  the  same  for  supervisors  and  subordinates  (£(2,52)  *  .21,  £  =  .82  for  the 
rating  scale  by  incumbent  group  interaction).  All  of  this  is  sensible.  The 
coordinator  positions  should  be  rated  higher  on  Frequency,  Importance,  and 
Difficulty.  The  similarity  of  the  pattern  of  ratings  for  coordinators  and 
subordinates  is  further  evidence  that  their  jobs  are  similar  in  type,  although 
different  in  degree. 

To  summarize,  there  is  appreciable  similarity  between  the  jobs  of  the 
supervisory  and  subordinate  submode  operators.  That  similarity  is  primarily 
reflected  by  the  Frequency  and  Importance  rating  scales.  The  analysts 
captured  some  of  the  aspects  of  the  job  described  by  the  incumbents,  but  they 
appear  to  have  interpreted  the  Importance  scale  like  incumbents  interpreted 
the  Difficulty  scale. 
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Relations  Among  the  Frequency,  Importance,  and  Difficulty  Scales. 

Tables  1  and  2  indicate  that  the  Frequency  and  Importance  scales  conveyed 
very  similar  information,  as  interpreted  by  the  incumbents.  Perhaps  job 
elements  which  were  frequent  were  also  important  on  each  occasion  they  were 
done.  Another  possibility  is  that  the  description  of  the  scales  (see 
Appendix  A)  did  not  give  the  incumbents  adequate  information  to  differ¬ 
entiate  Frequency  from  Importance.  In  any  case,  the  scales  were  redundant 
in  this  application.  If  this  finding  were  repeated  then  one  of  these  scales 
might  be  deleted  from  such  job  analysis  questionnaires. 

The  Difficulty  scale  was  measuring  something  different  from  Frequency/ 
Importance  in  this  analysis.  There  wasn't  as  much  agreement  between  incumbent 
groups  about  the  difficulty  of  various  job  elements  in  the  questionnaire  as 
there  was  about  Frequency /Importance .  Nonetheless,  the  Difficulty  ratings 
were  somewhat  in  agreement  between  incumbent  groups.  The  aspect  of  the  job 
measured  by  the  Difficulty  scale  is  indicated  by  the  analysis  of  Table  2  to 
be  unrelated  to  the  aspect  measured  by  the  Frequency  and  Importance  scales. 
Hence,  the  Difficulty  scale  seems  to  add  information  to  the  job  analysis  that 
was  not  available  to  the  usual  Importance  scale.  In  contrast,  the  Frequency 
scale  does  not  seem  to  add  much  new  information. 

Job  Elements  Which  Characterize  the  Aegis  Submodes 

The  job  elements  which  characterize  the  submodes  are  the  elements  which 
received  extreme  ratings  from  the  incumbents.  Knowing  which  elements  were 
rated  high  or  low  for  frequency,  importance,  and  difficulty  should  give  one 
an  understanding  of  the  job.  In  addition,  extreme-rated  job  elements  should 
fall  into  certain  patterns  for  optimum  system  performance.  For  example, 
elements  that  occur  with  high  frequency  should  be  of  low  difficulty  if  the 
system  is  to  perform  reliably.  Frequent,  difficult  elements  are  areas  of 
possible  system  improvement.  Elements  related  to  equipment  use  should  have 
low  difficulty  if  the  equipment  is  properly  designed  and  used.  Furthermore, 
elements  which  are  frequent  and  important  may  suggest  traits  of  successful 
operators.  Operators  with  these  traits  may  be  obtained  by  training  or  by 
personnel  selection. 

Table  3  displays  the  elements  of  the  Aegis  submode  operator* s  job  which 
received  extreme  ratings.  The  elements  are  listed  separately  in  order  of 
Frequency,  Importance,  and  Difficulty  ratings.  Of  course,  the  extreme  ratings 
would  not  be  of  Interest  if  there  were  little  spread  of  the  ratings,  or  little 
agreement  among  the  Incumbents.  These  factors  were  taken  into  account  in  a 
statistical  (Newman -Keu Is)  procedure,  and  the  elements  listed  in  Table  3  are 
those  that  have  ratings  that  are  dispersed  beyond  mere  chance  difference  from 
at  least  one  other  element  at  the  opposite  extreme  of  the  rating  scale.  The 
Newman-Keuls  procedure  (Kirk,  1968)  is  constructed  so  that  considering  all 
pairs  of  ratings  within  one  scale  the  chance  of  incorrectly  declaring  any  two 
of  the  ratings  to  be  different  is  less  than  5  in  100. 

Study  of  Table  3  reveals  some  interesting  patterns.  For  instance,  use  of 
the  foot-operated  communication  switch  was  rated  as  the  most  frequent  element, 
and  also  as  the  least  difficult.  This  type  of  result  indicates  good  design  of 
the  man-machine  interface.  This  is  also  indicated  by  the  low  difficulty  rating 
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Table  3:  Job  elements  with  High  or  Low  Ratings 

_ Rating  Scales _ 

Frequency  Importance  Dif  f icul ty 


(highest) 

24. 

Use  of  foot  operated 
controls  (4.4) 

23. 

Use  of  activation 
controls 

28. 

Advising  others  (4.3) 

29. 

Routine  comras/not 
advising 

4. 

Use  of  verbal  infor¬ 
mation 

20. 

Working  under 
distractions 

High 

7. 

Monitoring  events 
(3.4) 

19. 

Attending  to  detail 

1 . 

Use  of  quantitative 
visual  information 

5. 

Recognition  of 
objects,  events, 
processes 

9. 

Decision  making 

6. 

Watching  for  infre¬ 
quent  events  (3.0) 

23. 

Use  of  activation  20. 

Working 

controls  (4.3) 

under 

4. 

Use  of  verbal 

distrac 

information 

tions 

19. 

Attending  to  detail 

(2.9) 

24. 

Use  of  foot-operated 
control  (3.7) 

29. 

Routine  coraras/not 
advising 

25. 

Use  of  trackball 

7. 

Monitoring  events 

9. 

Decision  making 
(3.6) 

6. 

Watching  for 
infrequent  events 

5. 

Recognition  of 
objects,  events, 
processes 

20. 

Working  under  dis¬ 
tractions 

18. 

Responding  under 
time  pressure  (3.2) 

(lowes  t) 


Low 


27. 

Writing  (1.1) 

27. 

Writing  (1.1) 

24. 

Use  of  foot 

17. 

Use  of  mathematics 

2. 

Use  of  visual 

operated 

2. 

Use  of  visual  infor¬ 

information  about 

controls 

mation  about  natural 

natural  features 

(1.0) 

features 

17. 

Use  of  mathematics 

25. 

Use  of 

10. 

Reasoning  (1.6) 

15. 

Copying  or  trans¬ 

trackball 

11. 

Planning  and  scheduling 

cribing 

27. 

Writing 

15. 

Copying  or  transcribing 

10. 

Reasoning  (2.0) 

(1.3) 

13. 

Analyzing  information 

25. 

Use  of  trackball 

30. 

Non-routine  communication 

22. 

Use  of  keyboard  devices 

(2.3) 

Elements  included  in  this  table  were  rated  statistically  significant  higher  or 
lower  (JP  < . 05 )  than  at  least  one  other  job  element  in  the  same  rating  scale. 
Mean  ratings  on  a  5-step  scale  are  given  in  parentheses  for  marker  elements. 

In  this  table,  each  element  description  is  preceded  by  its  number  from  our 
questionnaire  (Appendix  A) . 


1  i 


Job  Analysis /Aegis  Submode  System 
8 


of  trackball  use,  and  writing.  Working  under  distractions  is  a  frequent,  dif¬ 
ficult  element  which  may  be  a  key  to  further  system  improvement.  Further  work 
might  be  done  to  find  out  what  types  of  distractions  are  being  referred  to, 
and  what  might  be  done  to  reduce  their  frequency  and  difficulty.  If  we  turn 
our  attention  to  the  apparent  demands  of  the  task,  we  find  that  the  frequent, 
important  job  elements  heavily  involve  communication  (advising,  routine  com¬ 
munications  /not  advising,  use  of  verbal  information,  use  of  foot-operated  con¬ 
trols  (communication  switch)).  The  watchstander f s  duties  of  monitoring  events, 
recognition  of  events,  and  decision  making  were  also  rated  high  on  Frequency 
and  Importance.  A  third  area  of  task  demand  is  the  conflicting  elements  of 
attending  to  detail  and  working  under  distractions.  It  is  remarkable  that  men¬ 
tal  operations  such  as  analyzing  information,  reasoning,  use  of  mathematics,  and 
planning  and  scheduling  are  relatively  infrequent  and  unimportant  in  this  job. 
These  findings  may  be  related  to  the  limitations,  described  earlier,  of  the  data. 
They  should  be  reexamined  if  an  opportunity  arises  to  analyze  the  job  of  Aegis 
submode  operators  in  a  more  typical  context.  In  general.  Aegis  submode  operators 
could  be  selected  or  trained  for  efficient  communication,  alert  watchstanding, 
and  ability  to  attend  to  detail  while  working  under  distractions. 

Typical  Sequences  of  Action  in  Operation  of  Aegis  Submodes 

The  job  element  part  of  the  questionnaire  provided  valuable  Information 
about  what  is  done  on  the  job,  but  it  tells  us  little  about  when  it  is  done. 

The  sequence  of  job  elements  can  also  be  useful  information.  For  example, 
knowing  the  sequence  of  job  elements  can  aid  an  equipment  designer  to  con¬ 
figure  displays  and  controls  so  that  temporally  adjacent  job  elements  involve 
spatially  adjacent  equipment.  Training  is  likely  to  be  more  effective  if  it 
includes  elements  of  the  job  In  the  correct  sequence  indicated  by  job  analysis. 
Finally  personnel  tests  should  be  more  job-related  if  they  include  the  actual 
timing  of  elements  of  the  job. 

Seven  of  the  sixteen  incumbents  reponded  to  the  last  part  of  the  question¬ 
naire,  which  dealt  with  natural  sequences  of  job  elements  of  Aegis  submode 
operation.  The  incumbents  indicated  which,  if  any,  job  element  typically 
followed  each  other  element.  The  data  were  cast  in  the  form  of  a  matrix. 

Table  4  shows  the  number  ,  of  times  the  respondents  said  that  element  j 
(matrix  column)  was  followea^by  element  i  (matrix  row).  The  matrix  showed 
that  two  elements  often  follow  a  wide  variety  of  other  elements.  The  two 
common  end  points  of  many  element  sequences  were:  element  23,  "use  of  acti¬ 
vation  controls  (e.g.,  VAB)"  and  element  4,  ,fuse  of  (non  visual)  verbal 
information  related  to  the  job  (e.g. ,  verbal  instructions,  orders,  requests, 
etc.  that  are  relevant  to  the  job)".  A  common  beginning  for  sequences  was 
element  1,  "use  of  visual  information  that  tells  you  how  much  of  something 
there  is  or  should  be".  Two  common  transitions  were  to  element  23  from 
elements  1  or  9,  "decision  making:  choosing  from  among  a  variety  of  alter¬ 
natives,  taking  into  consideration  the  consequences  of  the  choice".  These 
action  sequences  are  depicted  in  Figure  1 . 

This  information  suggests  some  practical  actions.  The  transition  from 
element  1  to  element  23  suggests  that  quantitative  visual  displays  should  be 
located  next  to  appropriate  activation  controls,  so  that  it  would  be  easy  to 
shift  attention  from  one  to  the  other.  Furthermore,  training  should  involve 
the  sequences  of  elements  discussed. 
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The  poor  response  rate  to  this  part  of  the  questionnaire  (only  7  of  16 
Incumbents  responded)  indicates  that  it  needs  to  be  reworded  to  make  respond¬ 
ing  easier.  The  information  obtained  about  typical  action  sequences  appears 
to  be  worthwile. 

Educ at  ion  L  Tra L  n in g ,  a  ml  Upda t ing  of  Aegi s  Submode  Operators 

Questions  2,  3,  and  4  of  the  questionnaire  measures  incumbents1  percep¬ 
tions  regarding  education,  experience,  and  training.  Figure  2  shows  the 
frequency  distrlbut ion  of  responses  to  question  2  regarding  the  level  of 
formal  education  the  operators  believed  was  necessary  to  operate  their  sub¬ 
mode.  The  mean  was  12  years  with  a  standard  deviation  of  3.87  years.  The 
distribution  was  negatively  skewed  due  to  one  subject  who  responded  that 
formal  education  was  unnecessary.  In  general,  though,  the  operators  thought 
that  at  least  a  high  school  education  was  required  and  some  felt  some  post- 
high-school  preparation  was  necessary.  As  Figure  2  indicates,  the  subor¬ 
dinates  were  somewhat  more  varied  in  their  opinions  than  were  the  coordinators, 
although  this  could  be  due  to  the  smaller  number  of  respondents  for  that  group. 

In  terms  of  on-the-job  training  and  experience  thought  necessary  (question 
3),  the  operators  were  extremely  variable  in  their  responses  with  a  mean  period 
of  7.88  weeks  of  training  and  a  standard  deviation  of  6.72  weeks  (see  Fig  3). 
The  variability  in  this  response  could  be  due  to  several  sources  such  as  vari¬ 
ability  in  the  diff iculty /complexity  of  the  various  positions  and/or  differ¬ 
ences  in  ability  of  the  subjects.  The  operators  perceived  an  average  of  9.75 
hours  spent  per  month  in  studying  operating  procedures,  doctrine,  etc.,  as 
necessary  to  be  a  proficient  operator,  with  a  standard  deviation  of  3.12  hrs. 
(see  Figure  4).  Due  to  the  free  response  format  of  this  item,  the  information 
cannot  be  as  neatly  summarized  as  the  other  items.  The  variability  in 
responses  to  this  question  seemed  to  be  related  to  situational  factors  that 
went  unspecified  in  most  of  the  respondents’  answers. 

Even  disregarding  the  few  highly  atypical  responses,  the  profiles  on 
questions  3  and  4  (see  Figures  3,  4)  suggests,  most  strongly,  that  the 
operators  are  either  highly  variable  in  the  ability  they  bring  to  the  position, 
or  the  positions  themselves  are  rather  variable  in  their  related  demands. 

This  latter  idea  may  have  some  merit,  since  the  correlations  between  operator 
groups  were  as  low  as  .475  (for  difficulty);  not  a  poor  correlation  but  still 
not  accounting  for  a  large  amount  of  variance  of  difficulty. 


The  answer  to  question  5  regarding  what  degree  the  safety  of  others 
depends  on  correct  action  of  the  operators  varied  from  ’none  in  this  envi¬ 
ronment’  to  ’critical,  depending  on  the  environment  and  ship  situation* • 
The  safety  implications  of  action  by  subraode  operators  appear  to  be  widely 
variable . 


COMMAND/COORDINATOR 
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FIG  2  BDUCATION  MOUMBD  BOR  ABQIB  BUBMOOB  OPERATORS  (IS  VRB:NIOH  SCHOOL) 


Command/Coordinator 
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FIG  3  FREQUENCY  OF  RESPONSES  OF  OPERATOR  PERCIEVED  TRAINING  AND  ON  THE 
JOB  EXPERIENCE  NECESSARY  FOR  OPERATOR  PROFICIENCY  (Quas.  #3)  ns  IS 


FIG  4  FREQUENCY  OF  RESPONSES  OF  OPERATOR  PERCIEVED  TIME  NEEDED 
UPDATING  JOB  KNOWLEDGE  (Qu«s.+4)  ns16 
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A  job-analysis  questionnaire  was  responded  to  by  16  Inc unbent  Aegis  sub¬ 
mode  operators.  Their  responses  were  the  basis  for  analyses  related  to  com- 
nonallty  among  submodes,  job  elements  and  sequences  typical  of  submode 
operation,  and  the  educational  and  training  requirements  of  the  submodes. 

It  Is  concluded  that  there  is  considerable  commonality  among  the  submodes. 

Job  elements  typical  of  submode  operation  Involve  communication,  alert 
watchstanding,  and  attending  to  detail  while  working  under  distractions. 

Typical  sequences  of  job  elements  end  with  the  use  of  activation  controls 
(e.g.,  VAB)  ,  or  verbal  orders  /instruct  ion /requests,,  preceded  by  use  of  visual 
displays  or  decision  making  without  reference  to  an  external  Information 
source.  Educational  requirements  for  submode  operation  were  a  high-school 
education,  on  the  average.  Training  time  was  extremely  variable  with  a  mean 
of  about  eight  weeks.  About  10  hours  of  study  per  month  was  perceived  to  be 
necessary  to  maintain  proficiency  In  operating  procedures,  doctrine,  etc. 

Implications  for  Performance  Testing  under  Environmental  Stress 

Although  a  job  analysis  such  as  this  one  can  have  many  uses,  the  authors9 
primary  concern  was  to  learn  something  about  a  critical  job  in  the  Navy  so 
that  human  performance  experiments  could  be  made  more  relevant  to  performance 
In  the  fleet.  Our  human  performance  research  is  being  done  at  the  Naval  Bio- 
dynamlcs  Laboratory,  tfilch  has  facilities  for  carefully  controlled  simulations 
of  ship  motion  (heave),  vibration,  and  crash  impact.  We  would  like  to  show 
what  aspects  of  human  performance  are  affected  by  these  environmental 
stressors,  and  which  are  not.  For  example,  the  best  evidence  Is  that  ship 
motion  can  affect  the  speed  and  accuracy  of  continuous  whole-arm  movements 
(an  effect  which  Is  Independent  of  nausea);  vibration  can  effect  visual 
resolution  and  fine  manual  facility;  and  crash  Impact  can  affect  overall 
responsiveness,  and  memory.  Whether  or  not  these  findings  are  useful  to  the 
Navy  depends  upon  the  extent  to  which  Navy  jobs  require  the  affected  skills 
and  abilities.  For  Instance,  Aegis  submode  operation  would  be  adversely 
affected  by  stressors  which  Interfere  with  the  most  characteristic  job 
elements  and  job  sequences  of  submode  operation.  Characteristic  job  elements 
are:  use  of  foot  operated  controls,  use  of  activation  controls  (such  as  pushing 
a  VAB  with  a  continuous  whole-arm  movement),  use  of  verbal  Information,  advising 
others,  attending  to  details,  and  routine  communication.  A  characteristic 
sequence  Involves  use  of  visual  Information  or  decision  making,  followed  by  use 
of  activation  controls.  Conversely,  Aegis  submode  operation  should  be  little 
affected  by  stressors  which  Interfere  with  relatively  Infrequent  and  unimpor¬ 
tant  elements  such  as  writing,  use  of  mathematics,  reasoning,  or  use  of  visual 
information  about  natural  features.  This  information  will  help  us  to  design 
our  human  performance  experiments  for  maximum  relevance  to  the  fleet. 


Job  Analysis /Aegis  Submode  Syst< 
16 


REFERENCES 

Kirk,  R.  E.  Experimental  design:  Procedures  for  the  behavioral  sciences. 

Belmont ,  California:  Brooks /Cole,  1968.  ""  ~  " 

McCormick,  E.  J.  Job  analysis:  Methods  and  applications.  New  York:  AMACCM. 
1979. 

Shannon,  R.  H.  and  Carter,  R.  C.  A  comparison  of  tactical  naval  work  stations 
within  air  and  sea  environments.  Preprints  of  the  52nd  Annual  Scientific 
Meeting  of  the  Aerospace  Medical  Association,  San  Antonio,  May  1981, 
238-239. 


I 

i 


1 


appendix  a 

aegis  submode  questionnaire 


AEGIS  SUBMODE  QUESTIONNAIRE 


1.  Which  submode  do  you  operate? 


2.  What  level  of  formal  education  is  needed  to  operate  this  submode? 
_  years* 


3.  Given  the  needed  amount  of  formal  education,  how  much  training  and  on-the 
job  experience  does  an  operator  need  to  become  proficient? 
weeks* 


4.  How  much  time,  if  any,  must  the  proficient  operator  spend  undating  job 
knowledge  (e.g.,  studying  operating  procedures,  doctorine,  etc.)? 
_  hrs.  per  month. 


5.  To  what  degree  does  the  safety  of  others  depend  on  correct  action  of  the 
operators? 


Please  rate  the  following  actions  in  terms  of : 


*  How  frequently  they  apply  to  operation  of  this  submode. 

*  Importance  to  mission  success,  without  regard  to  frequency. 

*  Difficulty  of  one  instance  of  the  action.  This  is  the  typical  amount 
of  attention  and  effort  required  to  accomplish  the  action  once,  using  this 
8ubmode*  Rate  the  difficulty  of  accomplishing  the  action  once;  do  not  con¬ 
sider  frequency.  Difficulty  is  related  to  complexity  of  an  action  or  the 
amount  of  training  required  to  perform  it. 


Use  the  following  five-point  scales  to  rate  each  item  RELATIVE  TO  ALL 
OTHER  ITEMS.  For  example,  every  action  may  be  of  critical  importance,  but 
some  are  more  critical  than  others  in  operation  of  this  submode.  An  action 
that  is  more  critical  than  about  half  of  all  actions  would  be  rated  3  on  the 
importance  scale,  indicating  moderate  Importance  relative  to  other  actions. 
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I*  Frequency  relative  to  other  Items  on  this  Inventory 


0  -  Does  not  apply 

1  -  Very  infrequent 

2  -  Occasional 

II •  Importance  to  mission  success,  without 

0  -  None  (no  impact  on  mission 

success) 

1  -  Little 

2  -  Some 

III.  Average  difficulty,  without  regard  to 

0  -  Done  without  noticing 

I  -  Easy  (Can  be  done  while 

doing  other  things) 


3  -  Moderate 

4  -  Often 

5  -  Very  often 

regard  to  frequency 

3  -  Moderate 

4  -  High 

5  -  Critical  (Success  of  the  mission 

depends  upon  correct  action) 

frequency 

3  -  Moderately  difficult 

4  -  Difficult  (requires  undivided 
attention  and/or  effort) 

Very  difficult  (complex 
action,  possibly  requiring 
much  experience  and/or 
training) 


2 


Slightly  difficult 


5 
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Input  to  Operator 


Frequency 


Importance  Difficulty 


1.  Use  of  visual  information  that 
tells  you  how  much  of  something 
there  is  (or  should  be) (e.g., 
number  of  missiles  in  a  magazine) 


2.  Use  of  visual  information  about 
natural  features  (e.g.,  weather, 
land,  sea) 


3.  Use  of  visual  information 
about  man-made  features  (e.g., 
ships,  cities,  aircraft) 


4.  Use  of  (non  visual)  verbal 
inforraaton  related  to  the  job 
(e.g.,  verbal  instructions,  orders, 
requests,  etc .  that  are  relevant  to 
the  job) 


5.  Recognition  of  objects,  events, 
or  processes  (e.g.,  recognizing  a 
situation  as  described  in  the 
doctorine) 


6.  Watching  for  infrequent  events 
(e.g.,  equipment  failure) 


7.  Monitoring  continually  changing 
events  (e.g.,  the  deployment  of 
hostile  and  friendly  forces) 


8.  Other  actions  typical  of  this 
submode  that  come  before  mental 
processing  and  response.  Specify: 


A-5 


Mental  Processing  of  Combat  Information 


Frequency  Importance  Difficult^ 


9.  Decision  making:  choosing  from  among 
a  variety  of  alternatives,  taking  into 
consideration  the  consequences  of  the 
choice  (e.g.,  initiating  an  attack) 


10.  Reasoning:  problem  solving  by  using 
rules  or  principles  to  draw  conclusions 
(e.g.,  using  the  three-minute  rule) 


11.  Planning  and  scheduling  (planning 
the  order  of  engaging  elements  of  a 
hostile  force) 


12.  Combining  information  from 
multiple  sources  (e.g.,  integrating 
multi-sensor  information) 


13.  Breaking  down  information  into 
component  parts  to  find  underlying 
principles  or  facts  (e.g.,  analyzing 
an  EW  signal) 


14.  Gathering,  grouping,  classifying, 
or  otherwise  arranging  information 
(e.g.,  grouping  contacts  as  to  prob¬ 
able  ship  or  aircraft  types) 


13,  Copying  or  transcribing  information 
(e.g.,  writing  down  a  vergal  transmission) 


16.  Recalling,  after  a  brief  time,  infor¬ 
mation  that  was  memorized  while  doing  the 
job  (e.g.,  remembering  the  location  of 
various  contact  numbers) 


17.  Using  mathematics  such  as  counting, 
arithmetic,  fractions,  geometry,  trig¬ 
onometry,  algebra,  etc.  (e.g.,  counting 
the  number  of  neutral  aircraft,  or 
solving  a  relative  motion  problem) 


18*  Responding  under  time  pressure 
(urgent  deadlines,  rush  jobs) 


19.  Attending  to  detail,  making  sure 
nothing  is  left  undone 


20.  Working  under  distractions 
(interruptions,  two  things  requiring 
attention  at  once) 


21.  Other  aspects  of  mental  processing 
of  Combat  Information.  Specify: 


Output  from  Operators 

22.  Use  of  keyboard  devices 

23.  Use  of  activation  controls  (e.g., 

VAB) 

24.  Use  of  foot  operated  controls 

25.  Use  of  trackball 


26.  Use  of  other  controls.  Specify: 
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27.  Writing  (e.g.,  making  log  entries) 

28.  Advising  others  (e.g.,  counseling 
or  guiding  others  for  the  purpose  of 
solving  problems) 


29.  Routine  job-reLated  communication 
(other  than  advising)  (e.g.,  communi¬ 
cation  in  standard  format  and 
vocublary) 


30.  Non-routine  job-related  communication 
(other  than  advising)  (e.g.,  communication 
that  requires  deviation  from  standard 
format  and  vocabulary) 


31.  Other  operator  output.  Specify: 


Please  indicate  common  sequences  of  actions  by  using  action  numbers  1-31 
from  the  preceding  part  of  this  questionnaire.  Below  are  listed  the  numbers 
1-31  to  represent  the  first  action  in  a  sequence.  If  operation  of  your  sub¬ 
mode  often  requires  following  one  of  these  actions  with  a  second  action, 
please  write  the  number  of  the  second  action  after  the  number  of  the  first 
action.  For  example,  if  after  making  a  decision  (action  9)  you  often  press 
a  VAB  (action  23),  then  mark  23  after  the  number  9  below. 


1 

9 

17 

25 

2 

10 

18 

26 

3 

11 

19 

27 

4 

12 

20 

28 

5 

13 

21 

29 

6 

14 

22 

30 

7 

15 

23 

31 

8 

16 

24 

